Streaming End-to-End Multi-Talker Speech Recognition

End-to-end multi-talker speech recognition is an emerging research trend in the community due to its vast potential applications such as conversation and meeting transcriptions. To best of our knowledge, all existing works are constrained offline scenario. In this work, we propose Streaming Unmixing Recognition Transducer (SURT) for end-to-end recognition. Our model employs Recurrent Neural Network (RNN-T) backbone that can meet various latency constraints. We study two different architectures based on a speaker-differentiator encoder mask respectively. train model, investigate widely used Permutation Invariant Training (PIT) approach Heuristic Error Assignment (HEAT) approach. Based experiments publicly available LibriSpeechMix dataset, show HEAT achieve better accuracy compared with PIT, SURT 150 milliseconds algorithmic constraint compares favorably sequence-to-sequence baseline terms accuracy.